There is a growing number of imaging applications where the viewer has the perception of being able to move about a virtual environment. One method of developing such a virtual environment is to capture a plurality of images, which can be combined into 360.degree. panoramic view. Panoramic images can be considered as capturing an image as if the film were wrapped in a cylinder. For applications such as virtual reality, a portion of the image is transformed to appear as if the image were captured with a standard photographic system. However, since optical systems are not easily built which capture images on cylindrical photoreceptors, a variety of methods have been developed to provide the functionality.
A direct means for capturing panoramic images, uses a rotating vertical slit camera, such as the Roundshot and Cirkut Camera. Such a camera captures the panoramic image by exposing film through a slit and then rotating the camera while moving the slit to expose a panoramic image on the film. These systems are rather expensive, and are not commonly used. IPIX and BeHere capture panoramic images with mirrors using standard photographic film. The advantage of these systems is that it captures the entire image at one instance.
Another means for accomplishing this task is to have multiple cameras which are synchronized and calibrated in order to perform the splicing to form the panoramic view without resorting to rotating a camera for individual images. Such a system was developed by Vishvjit Nalwa (see U. S. Pat. No. 5,539,483, "Panoramic Projection Apparatus," issued Jul. 23, 1996). This system uses several cameras and permits the simultaneous capture of a panoramic image.
Recently another method has been developed where a camera system is mounted on a pivoting tripod and the camera is incrementally rotated and individual images are captured. Then the images are digitized and spliced into a continuous 360.degree. panorama (see S. E. Chen and G. S. P Miller, "Cylindrical to planar image mapping using scanline coherence," U.S. Pat. No. 5,396,583, issued Mar. 7, 1995). This approach is used by the software application known as "Photo Vista" sold by Live Picture, Inc. Portions of the panoramic images are shown to the viewer in such a manner as to give the perception of moving about the virtual environment.
One feature of rotating a camera systems is that the position of the axis of rotation is at the rear nodal point of the camera's optical system. This is well-known (see S. Ray, Applied Photographic Optics, Lenses and Optical Systems for Photography, Film, Video and Electronic Imaging, The Bath Press, Avon, England, 1994, p. 42) and is highly desirable for any panoramic system.
The position of the rear nodal point in the camera optics is used in the design of conventional panoramic camera mounts. Such a design is included in the DC-220VR camera kit produced by Eastman Kodak Company, which may be used with the DC-220 camera to capture panoramic images. The system also locks the camera to proper position to eliminate misalignment which might occur with a conventional camera mount system. Positioning the rear nodal point on the axis of rotation assures that the best possible results are obtained.
Because conventional panoramic images do not have range associated with the objects in the scene there are many potential applications of virtual-reality which are not accomplished easily. One such application is the ability to introduce objects synthetically into a panoramic image and interact with the image as one might like. For instance, if the objects are a distance d from the camera and a synthetic object is desired to be placed midway and have the zooming property of virtual images to operate in a manner appearing normal, the range must be known. Also, if synthetic objects are to interact with real objects in the image, the range information is critical.
Applying panoramic images for virtual reality is very attractive as it is easily implemented and offers a means of providing high quality images in a real-time imaging system. However, since the panoramic image capture systems do not acquire images from different vantage points, it is unreasonable to expect the system to estimate the range of the objects to the image capture point. Estimating range information can be accomplished in many ways, though a common and well-known method is stereo image pairs.
Conventionally, stereo images are captured using two optical systems having parallel optical axis. The plane formed by the optical axis and vector between image centers being parallel to the ground. The reason for this is that stereo images are typically used to give the viewer a perception of three-dimensionality, and the human visual system has this arrangement. There have been systems producing stereo panoramic images, (see Huang and Hung, Panoramic Stereo Imaging System with Automatic Disparity Warping and Seaming, Graphical Models and Image Processing, Vol. 60, No. 3, May, 1998, pp. 196-208), however, this system uses a classical side-by-side stereo system as the intent is to utilize the stereo images for a human viewer, and not to estimate the depth of objects. One problem of the side-by-side approach is that panoramic images are best captured when the axis of rotation is at the rear-nodal point of the optical system. In a conventional side-by-side configuration this is geometrically impossible. As a result, at least one of the panoramic images is sub-optimal.
The ability to estimate range from a pair of stereo images is well known and references to this can be found in many standard textbooks.
Difficulties with stereo imaging are in matching corresponding image points in two images. Another problem is that the range estimates are partially based upon the disparity, or distance between the focal points of the two cameras. For a classical stereo system the problem of the disparity is that better depth estimates can be attained when the disparity increases, but as the disparity increases the number of image points in correspondence decreases, which diminishes the value of the system. A general "rule-of-thumb" is that the disparity should be at least 1/10 of the range to be estimated. Hence if an object is in the far field, a wider disparity is preferable. The need for a large disparity aggravates the problem noted above. There is a need therefore for an improved stereoscopic panoramic photographic method and apparatus.